PSYC 3032 M
Mixing Categorical and Continuous Predictors
group):\[\hat{Reading \ score}_i = {\color{deeppink} {\beta_0}} + {\color{darkcyan} {\beta_1}}{\color{darkgrey} {D1_i}} + {\color{gold} {\beta_2}}{\color{lightblue} {D2_i}}\]
\[\hat{Reading \ score}_i = {\color{deeppink} {6.68}} + {\color{darkcyan} {3.09}}{\color{darkgrey} {D1_i}} + {\color{gold} {1.09}}{\color{lightblue} {D2_i}}\]
\[\hat{Reading \ score}_i = {\color{deeppink} {6.68}} + {\color{darkcyan} {3.09}}{\color{darkgrey} {(DRTA \ vs. \ Control)_i}} + {\color{gold} {1.09}}{\color{lightblue} {(TA \ vs. \ Control)_i}}\]
Regrssion with a Categorical Predictor is a One-Way ANOVA
\[\hat{Reading \ score}_i = {\color{deeppink} {\beta_0}} + {\color{darkcyan} {\beta_1}}{\color{darkgrey} {D1_i}} + {\color{gold} {\beta_2}}{\color{lightblue} {D2_i}} = \\ \hat{Reading \ score}_i = {\color{deeppink} {6.68}} + {\color{darkcyan} {3.09}}{\color{darkgrey} {D1_i}} + {\color{gold} {1.09}}{\color{lightblue} {D2_i}}\]
What about including other predictors beyond a single categorical variable?
Given that the dummy coding approach discussed last lecture are analogous to a one-way ANOVA, we could think of an MLR model with at least one categorical predictor and at least one continuous predictor as an ANCOVA (Analysis of Covariance)!
So, an ANCOVA model is just a certain type of multiple regression model that includes both categorical and continuous predictors…
Typically, ANCOVA is used for comparing group means on an outcome variable while controlling for some continuous variable
If ANOVA/regression with a categorical variable is commonly presented as a method for comparing group means, ANCOVA is often presented as a method for comparing adjusted means across groups (AKA conditional means)
The interpretation of the slope associated with the categorical predictor will change slightly, but you’re already familiar with this change!
This will not be different than the difference between any other predictor in SLR when you move to MLR
QUICK EXAMPLE
Say we estimate a model where life satisfaction is regressed on different types of meditation interventions (e.g., control, mindfulness, mantra-based, and gratitude-based), conditioning on socio-economic status (SES)
Then, we can interpret the dummy-code representing one of the comparisons between mediation types as the mean difference in life satisfaction between, say, mindfulness and control, adjusted for SES (or holding SES constant)
But, there’s this other thing…
ANCOVA has an additional critical assumption which is known as homogeneity of regression, or parallelism
Parallelism means that the relationship between the continuous variable, \(X\), and the outcome, \(Y\), is assumed to be constant, or homogeneous, across the levels of the categorical predictor
Put another way, the traditional ANCOVA model assumes that there is no interaction between \(X\) and the categorical variable
But, if we reframe ANCOVA as multiple regression, it’s easy to relax this assumption by including an interactions term between \(X\) and the dummy-coded variables representing group membership (we will address interactions in Module 7)
Another way to think about the parallelism assumption from a regression framework is the assumption that your model is properly specified
Yes, the reading comprehension example, again!
Baumann et al. (1992) were actually interested in how the groups differed in their post-intervention reading test scores over and above any differences on a reading test score administered before the intervention
This is an example of a classic “Pre-Post” research question for which ANCOVA is often applied:
Our regression/ANCOVA model will provide estimates of the adjusted mean differences, controlling for pre-test score
Baumann et al. (1992) were actually interested in how the groups differed in their post-intervention reading test scores over and above any differences on a reading test score administered before the intervention
\[\hat{Post}_i=\beta_0 + \beta_1D1_i + \beta_2D2_i + \beta_3Pre_i\]
Using this ANCOVA approach, the omnibus, overall effect of the intervention variable is the joint effect of D1 and D2, taken together
To obtain this joint effect and its statistical significance, we can follow a hierarchical regression procedure
\(\text{Model 1}: \hat{Post}_i= {\color{deeppink} {\beta_0 + \beta_1Pre_i}}\) vs.
\(\text{Model 2 (ANCOVA)}: \hat{Post}_i= {\color{deeppink} {\beta_0 + \beta_1Pre_i}} + \beta_2D1_i + \beta_3D2_i\)
And, for the actual model comparison, the F test:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.5966478 1.1845062 -0.5037101 6.162502e-01
pretest1 0.6931872 0.1014697 6.8314735 4.205253e-09
as_factor(group)DRTA 3.6265538 0.7361861 4.9261371 6.553543e-06
as_factor(group)TA 2.0361644 0.7449616 2.7332475 8.161831e-03
2.5 % 97.5 %
(Intercept) -2.9644420 1.7711465
pretest1 0.4903523 0.8960222
as_factor(group)DRTA 2.1549387 5.0981689
as_factor(group)TA 0.5470074 3.5253214
What about Using Change Scores Instead of Conditioning on Pretest?
Another approach with pre-post designs, is to model the difference between post-test and pre-test scores such that the outcome variable is the difference betwe post and pre rather than posttest and control for pretest:
Although ANCOVA and change scores represent different approaches to handling the same research design, researchers may actually arrive at different conclusions if they were to use both approaches on the same dataset; this phenomenon is called Lord’s Paradox (Lord, 1967)
Lord’s original experiment was meant to evaluate how young men and women differ on weight change over the course of a semester
But, men obviously start at a much higher average weight than women
In Lord’s dataset, men and women do not change at all over time (mean change = ~0 in both groups). Here’s a simulated illustration…
Call:
lm(formula = change ~ gender, data = dat)
Residuals:
Min 1Q Median 3Q Max
-15.5233 -3.3977 0.6239 3.9303 13.8364
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.1692 0.5190 2.253 0.0254 *
genderMen 0.1209 0.7340 0.165 0.8694
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 5.19 on 198 degrees of freedom
Multiple R-squared: 0.0001369, Adjusted R-squared: -0.004913
F-statistic: 0.02711 on 1 and 198 DF, p-value: 0.8694
Call:
lm(formula = final ~ gender + initial, data = dat)
Residuals:
Min 1Q Median 3Q Max
-9.4746 -1.9902 0.1751 1.8293 8.0446
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 33.79411 1.67270 20.20 <2e-16 ***
genderMen 13.42333 0.79429 16.90 <2e-16 ***
initial 0.44538 0.02797 15.92 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 3.006 on 197 degrees of freedom
Multiple R-squared: 0.9463, Adjusted R-squared: 0.9457
F-statistic: 1734 on 2 and 197 DF, p-value: < 2.2e-16
Results
Using t test on change score, we do not conclude a difference in between men and women
Using ANCOVA, we conclude a difference between men and women in final weight controlling for initial weight
Change scores addresses the question, “What is the difference in weight change between men and women?”
ANCOVA answers, “Is there still a difference between men and women in their final weights, after accounting for where each individual started (initial weight)?”
Lord’s original experiment was meant to evaluate how young men and women differ on weight change over the course of a semester
ANCOVA Assumptions
The assumptions of OLS regression apply equivalently to models with discrete predictors
library(car)
mod2 <- lm(posttest1 ~ pretest1 + as_factor(group), data = read)
stud_resid <- rstudent(mod2) # Studentizing the model residuals
scatterplot(stud_resid ~ read$group, boxplot=FALSE)The assumptions of OLS regression apply equivalently to models with discrete predictors and the same diagnostic procedures presented in earlier modules can be used
Recall, LINE:
And, of course:
\(\hat{Post}_i= {\color{deeppink} {\beta_0}} + {\color{deeppink} {\beta_1D1_i}} + {\color{deeppink} {\beta_2D2_i}} + {\color{deeppink} {\beta_3Pre_i}}\) vs. \(\hat{Post}_i={\color{deeppink} {\beta_0}} + {\color{deeppink} {\beta_1D1_i}} + {\color{deeppink} {\beta_2D2_i}} + {\color{deeppink} {\beta_3Pre_i}} + \beta_4D1_i \times Pre_i + \beta_5D2_i \times Pre_i\)
read$group <- haven::as_factor(read$group) # Ensure the group variable is treated as a factor
interaction_mod <- lm(posttest1 ~ pretest1 * group, data = read) # By using * we automatically add all the terms, beta1 through beta5 in the model above!
no_int_mod <- lm(posttest1 ~ pretest1 + group, data = read) # the same as mod2 from earlier[1] 0.0233242
Analysis of Variance Table
Model 1: posttest1 ~ pretest1 * group
Model 2: posttest1 ~ pretest1 + group
Res.Df RSS Df Sum of Sq F Pr(>F)
1 60 347.97
2 62 365.43 -2 -17.461 1.5054 0.2302
ggplot(read, aes(x = pretest1, y = posttest1, color = group)) +
geom_smooth(method = "lm", se = TRUE, aes(fill = group), alpha = 0.25) + # Add linear regression lines with semi-transparent confidence bands
geom_point(size = 2, alpha = 0.6) + # Plot the points with slight transparency
labs(x = "Pretest Score", y = "Posttest Score", title="Regression Slopes by Group") +
theme(legend.position = "none") +
theme_classic()Module 6 (Part 2)